Building datum extraction from laser scanning data

ABSTRACT

A method, apparatus, system, and computer program product provide the ability to extract level information and reference grid information from point cloud data. Point cloud data is obtained and organized into a three-dimensional structure of voxels. Potential boundary points are filtered from the boundary cells. Level information is extracted from a Z-axis histogram of the voxels positioned along the Z-axis of the three-dimensional voxel structure and further refined. Reference grid information is extracted from an X-axis histogram of the voxels positioned along the X-axis of the three-dimensional voxel structure and a Y-axis histogram of the voxels positioned along the Y-axis of the three-dimensional voxel structure and further refined.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit under 35 U.S.C. Section 119(e) ofthe following co-pending and commonly-assigned U.S. provisional patentapplication, which is incorporated by reference herein:

Provisional Application Ser. No. 61/871,042, filed on Aug. 28, 2013, byYan Fu, entitled “BUILDING DATUM EXTRACTION FROM LASER SCANNING DATA,”attorneys' docket number G&C 30566.506-US-P1.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to point cloud data, and inparticular, to a method, apparatus, and article of manufacture forextracting the level and reference grid of floor plan information of abuilding from point cloud data.

2. Description of the Related Art

(Note: This application references a number of different publications asindicated throughout the specification by references enclosed inbrackets, e.g. [x]. Such references may indicate the first named authorand year of publication e.g., [Okorn et al. 2010]. A list of thesedifferent publications ordered according to these references can befound below in the section entitled “References.” Each of thesepublications is incorporated by reference herein.)

Building information models (BIM) are being increasingly used throughouta building's lifecycle in the architecture, engineering, andconstruction (AEC) industry. BIMs can be used for many purposes, fromplanning and visualization in the design phase, to inspection during theconstruction phase, and to energy efficiency analysis and securityplanning during the facility management phase. However, BIMs are oftennot available for most existing buildings. Further, the BIM createdduring the design phase may vary significantly from what was actuallybuilt. As a result, there is strong interest in creating BIMs of theactual as-built building.

Laser scanners are rapidly gaining acceptance as a tool forthree-dimensional (3D) modeling and analysis in the architecture,engineering, and construction (AEC) domain. With technologicaldevelopment/evolution, laser scanners are capable of acquiring rangemeasurements at rates of tens to hundreds of thousands of points persecond, at distances of up to a few hundred meters, and with ameasurement error on the scale of millimeters. These characteristicsmake them well suited for densely capturing the as-built information ofbuilding interiors and exteriors. Typically, laser scanners are placedin various locations throughout and around a building. The scans fromeach location are registered and aligned to form a point cloud in acommon coordinate system. Multiple scans are often needed to capture thepoint cloud of a whole building.

Currently, as-built BIMs are mostly created interactively from the pointcloud data generated by the laser scanners. However, this creationprocess is labor-intensive and error-prone. Thus, there is a lack ofresearch work and commercial software tools currently available forautomatically extracting building datum information from point clouds.

In most applications and software tools, floor plan modeling is achievedby first creating a horizontal slice of the environment [Li et al. 2011]and then using various two-dimensional (2D) geometric modeling methods[Nguyen et al. 2005], including RANSAC (RANdom SAmple Consensus),iterative end point fitting, and the Hough transform, to extract thelinear geometry in the horizontal slice. For example, Okorn et al.[Okorn et al. 2010] examines floor plan modeling of wall structures tocreate blueprints from terrestrial laser scanning points. The directionof gravity (the vertical direction) is assumed to be known. Atwo-dimensional histogram is created from the points projected onto theground plane. Linear structures from this histogram are then extractedusing a Hough transform.

However, a floor plan modeling method based on only one singlehorizontal slice of the environment does not take the whole buildinginterior environment information into consideration, which means someelements might be missing from the single slice. Thus, a single slicedoes not adequately represent the whole floor plan structure.Furthermore, it is difficult to determine the slice height mostappropriate for generating the floor plan and therefore it is difficultto automatically select the height of the slice. Moreover, a singleslice method does not filter out points obtained from the objects andclutter existing in the interior environment of the building, whichfurther prevents the generation of a clear and accurate floor plan map.For point cloud data captured by terrestrial laser scanners, wall pointsnear the floor surface are more likely to be obstructed by furniture andother clutter. Additionally, the wall points near the ceiling surfaceare likely to be obstructed by other MEP (mechanical, electrical, andplumbing) utilities or decorations.

On the other hand, a three-dimensional (3D) method first models theplanar wall, floor, and ceiling surface and then creates the levels andfloor plan information with a cross-section step. However, such a methodis computation-heavy and due to the existence of noise and outliers, thewall, floor, and ceiling surface cannot be modeled perfectly.

Other related works on floor plan modeling/mapping have mainly focusedon robotics research [Schroter et al. 2002]. Such floor plan maps areusually generated by robots equipped with laser scanners. The mainpurpose of the maps is for use in robotic navigation. Therefore, theresearch on generating these types of navigation maps do not place muchemphasis on being highly accurate or complete.

In view of the above, it is desirable to extract/determine the level andfloor plan information of a building from building point cloud data inan easy and efficient manner. Software tools are needed for processingpoint clouds to improve the ability to handle the enormous point cloudsproduced by laser scanners and to integrate the use of point cloud datainto BIM modeling software.

SUMMARY OF THE INVENTION

Embodiments of the invention provide a computer-implemented method forextracting level information and reference grid information from pointcloud data. To extract the levels and orthogonal reference grids from aterrestrial point cloud data of a building's interior, principal axisdirections are first estimated by extracting plane normal informationand then the point cloud data is transformed to make the building standupright with the front side as the main façade surface. To addressproblems commonly associated with non-uniform point density, thetransformed point cloud data is organized into a 3D voxel structure.Level information is then roughly extracted by detecting the peaks of ahistogram generated by projecting the voxels along the X-axis and thenrefined by a plane-sweeping method. To extract the orthogonal referencegrid information, the floor plan points are filtered by removing pointsbelonging to large horizontal objects and selecting only pointsbelonging to straight walls. After that, a histogram is generated bycounting the occupied cells of the floor plan points projected onto theX-axis and Y-axis respectively. Peaks indicated in the histogram act asrough location markers for the reference grids and a line sweepingmethod is then employed to refine the location of reference grids.

BRIEF DESCRIPTION OF THE DRAWINGS

Referring now to the drawings in which like reference numbers representcorresponding parts throughout:

FIG. 1 is an exemplary hardware and software environment used toimplement one or more embodiments of the invention;

FIG. 2 schematically illustrates a typical distributed computer systemusing a network to connect client computers to server computers inaccordance with one or more embodiments of the invention;

FIG. 3 is a flowchart illustrating the logical flow for performing alevel and floor plan information extraction process, in accordance withone or more embodiments of the invention;

FIGS. 4A-4C illustrate result comparisons of a building point cloudbefore and after principal axis adjustment, in accordance with one ormore embodiments of the invention. FIG. 4A depicts the front side of anillustrative building structure before principal axis adjustment. FIG.4B depicts the left side of the building after principal axisadjustment. FIG. 4C depicts the front side of the building afterprincipal axis adjustment;

FIGS. 5A and 5B depict level detection in the point cloud data of theillustrative building structure, in accordance with one or moreembodiments of the invention. FIG. 5A shows a histogram of voxelsprojected along a Z-axis. FIG. 5B illustrates the detection of threelevels (locations of flooring/ceiling);

FIG. 6 depicts the remaining floor plan points for the illustrativebuilding structure after floor, ceiling, and large horizontal objectsare removed, in accordance with one or more embodiments of theinvention;

FIG. 7 depicts a histogram of the number of voxels at the floor planpoints location of the illustrative building structure, in accordancewith one or more embodiments of the invention;

FIG. 8 depicts the floor plan points of the illustrative buildingstructure after filtering, in accordance with one or more embodiments ofthe invention;

FIG. 9 depicts the reference grids of floor plans of the illustrativebuilding structure, in accordance with one or more embodiments of theinvention; and

FIG. 10 depicts the extracted datum information from the building pointcloud data of the illustrative building structure, in accordance withone or more embodiments of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following description, reference is made to the accompanyingdrawings which form a part hereof, and which is shown, by way ofillustration, several embodiments of the present invention. It isunderstood that other embodiments may be utilized and structural changesmay be made without departing from the scope of the present invention.

Overview

For building information model (BIM) modeling software, two mainelements—levels and floor plans are critical for users authoring BIMs.This is the same case for when a user desires to create as-built BIMs ofexisting buildings from point cloud data because levels and floor planinformation provide the main conceptual model of the building.Embodiments of the present invention provide methods to automaticallyextract the level and floor plan information from laser scanning points.The input data mainly comes from terrestrial laser scanners. In certainembodiments, orthogonal reference grid information is extracted to aidin the authoring of floor plan information.

Although methods such as RANSAC (RANdom SAmple Consensus) and Houghtransforms have been proposed to extract 2D floor plan geometries fromfloor plan points, often it is still difficult to determine the accuratelocation of straight walls. One reason is that a wall has two faces andit is not clear which face has been scanned. Another reason is that thewall might not be well captured due to cluttering and occlusion factors.However, in common buildings, interior walls are usually designed withregular structures. Thus, extracting reference grids from extractedfloor plan points provides a global overview of the floor plan, whichcan act as a good reference for straight wall reconstruction.

To extract the levels and orthogonal reference grids from a terrestrialpoint cloud data of a building interior, the principal axis directionsof the point cloud data are first estimated with the extracted planenormal information. Then, the point cloud data is transformed to makethe building stand upright with the front side as the main facadesurface. To address the problems caused by non-uniform point density,the transformed point cloud data is organized into a three-dimensional(3D) voxel structure. Level information is then roughly extracted bydetecting the histogram peaks generated by projecting the voxels alongthe X-axis and then refining the level information using aplane-sweeping method. To extract the orthogonal reference gridinformation, the floor plan points are first filtered by removing pointsthat belong to large horizontal objects and selecting only points thatbelong to straight walls. Next, a histogram is generated by counting theoccupied cells of the floor plan points projected onto the X-axis andY-axis respectively. Histogram peaks act as rough location markers ofthe reference grids and a line sweeping method is then employed torefine the location of reference grids.

Hardware Environment

FIG. 1 is an exemplary hardware and software environment 100 used toimplement one or more embodiments of the invention. The hardware andsoftware environment includes a computer 102 and may includeperipherals. Computer 102 may be a user/client computer, servercomputer, or may be a database computer. The computer 102 comprises ageneral purpose hardware processor 104A and/or a special purposehardware processor 104B (hereinafter alternatively collectively referredto as processor 104) and a memory 106, such as random access memory(RAM). The computer 102 may be coupled to, and/or integrated with, otherdevices, including input/output (I/O) devices such as a keyboard 114, acursor control device 116 (e.g., a mouse, a pointing device, pen andtablet, touch screen, multi-touch device, etc.) and a printer 128. Inone or more embodiments, computer 102 may be coupled to, or maycomprise, a portable or media viewing/listening device 132 (e.g., an MP3player, iPod™, Nook™, portable digital video player, cellular device,personal digital assistant, etc.). In yet another embodiment, thecomputer 102 may comprise a multi-touch device, mobile phone, gamingsystem, internet enabled television, television set top box, or otherinternet enabled device executing on various platforms and operatingsystems.

In one or more embodiments, computer 102 may be coupled to, and/orintegrated with, a laser scanning device 134. Such a laser scanningdevice 134 is configured to scan an object or urban environment andobtain a digital representative of such an object/environment in theform of point cloud data that may be processed by the computer 102.Exemplary laser scanning devices 134 include terrestrial scanners (e.g.operated by hand or attached to a mobile device such as an automobile)as well as satellite based scanners.

In one embodiment, the computer 102 operates by the general purposeprocessor 104A performing instructions defined by the computer program110 under control of an operating system 108. The computer program 110and/or the operating system 108 may be stored in the memory 106 and mayinterface with the user and/or other devices to accept input andcommands and, based on such input and commands and the instructionsdefined by the computer program 110 and operating system 108, to provideoutput and results.

Output/results may be presented on the display 122 or provided toanother device for presentation or further processing or action. In oneembodiment, the display 122 comprises a liquid crystal display (LCD)having a plurality of separately addressable liquid crystals.Alternatively, the display 122 may comprise a light emitting diode (LED)display having clusters of red, green and blue diodes driven together toform full-color pixels. Each liquid crystal or pixel of the display 122changes to an opaque or translucent state to form a part of the image onthe display in response to the data or information generated by theprocessor 104 from the application of the instructions of the computerprogram 110 and/or operating system 108 to the input and commands. Theimage may be provided through a graphical user interface (GUI) module118A. Although the GUI module 118A is depicted as a separate module, theinstructions performing the GUI functions can be resident or distributedin the operating system 108, the computer program 110, or implementedwith special purpose memory and processors.

In one or more embodiments, the display 122 is integrated with/into thecomputer 102 and comprises a multi-touch device having a touch sensingsurface (e.g., track pod or touch screen) with the ability to recognizethe presence of two or more points of contact with the surface. Examplesof multi-touch devices include mobile devices (e.g., iPhone™, Nexus S™,Droid™ devices, etc.), tablet computers (e.g., iPad™, HP Touchpad™),portable/handheld game/music/video player/console devices (e.g., iPodTouch™, MP3 players, Nintendo 3DS™, PlayStation Portable™, etc.), touchtables, and walls (e.g., where an image is projected through acrylicand/or glass, and the image is then backlit with LEDs).

Some or all of the operations performed by the computer 102 according tothe computer program 110 instructions may be implemented in a specialpurpose processor 104B. In this embodiment, the some or all of thecomputer program 110 instructions may be implemented via firmwareinstructions stored in a read only memory (ROM), a programmable readonly memory (PROM) or flash memory within the special purpose processor104B or in memory 106. The special purpose processor 104B may also behardwired through circuit design to perform some or all of theoperations to implement the present invention. Further, the specialpurpose processor 104B may be a hybrid processor, which includesdedicated circuitry for performing a subset of functions, and othercircuits for performing more general functions such as responding tocomputer program instructions. In one embodiment, the special purposeprocessor is an application specific integrated circuit (ASIC).

The computer 102 may also implement a compiler 112 that allows anapplication program 110 written in a programming language such as COBOL,Pascal, C++, FORTRAN, or other language to be translated into processor104 readable code. Alternatively, the compiler 112 may be an interpreterthat executes instructions/source code directly, translates source codeinto an intermediate representation that is executed, or that executesstored precompiled code. Such source code may be written in a variety ofprogramming languages such as Java™, Perl™, Basic™, etc. Aftercompletion, the application or computer program 110 accesses andmanipulates data accepted from I/O devices and stored in the memory 106of the computer 102 using the relationships and logic that weregenerated using the compiler 112.

The computer 102 also optionally comprises an external communicationdevice such as a modem, satellite link, Ethernet card, or other devicefor accepting input from, and providing output to, other computers 102.

In one embodiment, instructions implementing the operating system 108,the computer program 110, and the compiler 112 are tangibly embodied ina non-transient computer-readable medium, e.g., data storage device 120,which could include one or more fixed or removable data storage devices,such as a zip drive, floppy disc drive 124, hard drive, CD-ROM drive,tape drive, etc. Further, the operating system 108 and the computerprogram 110 are comprised of computer program instructions which, whenaccessed, read and executed by the computer 102, cause the computer 102to perform the steps necessary to implement and/or use the presentinvention or to load the program of instructions into a memory, thuscreating a special purpose data structure causing the computer tooperate as a specially programmed computer executing the method stepsdescribed herein. Computer program 110 and/or operating instructions mayalso be tangibly embodied in memory 106 and/or data communicationsdevices 130, thereby making a computer program product or article ofmanufacture according to the invention. As such, the terms “article ofmanufacture,” “program storage device,” and “computer program product,”as used herein, are intended to encompass a computer program accessiblefrom any computer readable device or media.

Of course, those skilled in the art will recognize that any combinationof the above components, or any number of different components,peripherals, and other devices, may be used with the computer 102.

FIG. 2 schematically illustrates a typical distributed computer system200 using a network 202 to connect client computers 102 to servercomputers 206. A typical combination of resources may include a network202 comprising the Internet, LANs (local area networks), WANs (wide areanetworks), SNA (systems network architecture) networks, or the like,clients 102 that are personal computers or workstations, and servers 206that are personal computers, workstations, minicomputers, or mainframes(as set forth in FIG. 1). However, it may be noted that differentnetworks such as a cellular network (e.g., GSM [global system for mobilecommunications] or otherwise), a satellite based network, or any othertype of network may be used to connect clients 102 and servers 206 inaccordance with embodiments of the invention.

A network 202 such as the Internet connects clients 102 to servercomputers 206. Network 202 may utilize ethernet, coaxial cable, wirelesscommunications, radio frequency (RF), etc. to connect and provide thecommunication between clients 102 and servers 206. Clients 102 mayexecute a client application or web browser and communicate with servercomputers 206 executing web servers 210. Such a web browser is typicallya program such as MICROSOFT INTERNET EXPLORER™, MOZILLA FIREFOX™,OPERA™, APPLE SAFARI™, GOOGLE CHROME™, etc. Further, the softwareexecuting on clients 102 may be downloaded from server computer 206 toclient computers 102 and installed as a plug-in or ACTIVEX™ control of aweb browser. Accordingly, clients 102 may utilize ACTIVEX™components/component object model (COM) or distributed COM (DCOM)components to provide a user interface on a display of client 102. Theweb server 210 is typically a program such as MICROSOFT'S INTERNETINFORMATION SERVER™.

Web server 210 may host an Active Server Page (ASP) or Internet ServerApplication Programming Interface (ISAPI) application 212, which may beexecuting scripts. The scripts invoke objects that execute businesslogic (referred to as business objects). The business objects thenmanipulate data in database 216 through a database management system(DBMS) 214. Alternatively, database 216 may be part of, or connecteddirectly to, client 102 instead of communicating/obtaining theinformation from database 216 across network 202. When a developerencapsulates the business functionality into objects, the system may bereferred to as a component object model (COM) system. Accordingly, thescripts executing on web server 210 (and/or application 212) invoke COMobjects that implement the business logic. Further, server 206 mayutilize MICROSOFT'S™ Transaction Server (MTS) to access required datastored in database 216 via an interface such as ADO (Active DataObjects), OLE DB (Object Linking and Embedding DataBase), or ODBC (OpenDataBase Connectivity).

Generally, these components 200-216 all comprise logic and/or data thatis embodied in/or retrievable from device, medium, signal, or carrier,e.g., a data storage device, a data communications device, a remotecomputer or device coupled to the computer via a network or via anotherdata communications device, etc. Moreover, this logic and/or data, whenread, executed, and/or interpreted, results in the steps necessary toimplement and/or use the present invention being performed.

Although the terms “user computer”, “client computer”, and/or “servercomputer” are referred to herein, it is understood that such computers102 and 206 may be interchangeable and may further include thin clientdevices with limited or full processing capabilities, portable devicessuch as cell phones, notebook computers, pocket computers, multi-touchdevices, and/or any other devices with suitable processing,communication, and input/output capability.

Of course, those skilled in the art will recognize that any combinationof the above components, or any number of different components,peripherals, and other devices, may be used with computers 102 and 206.

Software Embodiments

Embodiments of the invention are implemented as a software applicationon a client 102 or server computer 206. Further, as described above, theclient 102 or server computer 206 may comprise a thin client device or aportable device that has a multi-touch-based display and that maycomprise (or may be coupled to or receive data from) a 3D laser scanningdevice 134.

FIG. 3 is a flowchart illustrating the logical flow for performing alevel and reference grid extraction process in accordance with one ormore embodiments of the invention.

At step 302, point cloud data is obtained (e.g., from a building scanusing a laser scanner). In one or more embodiments, the point cloud datacomprises laser scanning points for a building.

At step 304, the point cloud data is organized into a three-dimensional(3D) structure of voxels, the three-dimensional structure consisting ofan X-axis, Y-axis, and Z-axis. As used herein, a voxel represents avalue on a regular grid in 3D space.

At step 306, rough level information is extracted from a Z-axishistogram of the voxels positioned along the Z-axis of thethree-dimensional voxel structure.

At step 308, the extracted level information is refined.

At step 310, rough reference grid information is extracted from anX-axis histogram of the voxels positioned along the X-axis of thethree-dimensional voxel structure and a Y-axis histogram of the voxelspositioned along the Y-axis of the three-dimensional voxel structure.

At step 312, the extracted reference grid information is refined.

Details regarding the performance of one or more of the steps 302-312are described below.

Principal Direction Estimation

In one or more embodiments of the building datum extraction methoddescribed herein, a basic precondition is that the point cloud data isadjusted beforehand to make the building stand upright and that thefront of the building facade is parallel to the X-Z plane or the Y-Zplane. In most cases, this is already the orientation of the point clouddata. Otherwise this can be rectified by a coordinate systemtransformation using software provided by the laser scannermanufacturers. In one aspect of the invention, a method is provided totransform the point cloud data to make it stand upright. An importantfactor of the method is the estimation of the three principal directionsof the point cloud data. In one embodiment, the method comprisesextracting a principal direction of the point cloud data by determiningX-axis, Y-axis, and Z-axis directions of the point cloud data. The pointcloud data is then transformed such that a front face of the building isparallel to a plane defined by the X-axis and Z-axis or a plane definedby the Y-axis and Z-axis. The transformed point cloud data is thenorganized into the three-dimensional structure of voxels. In anotherembodiment, the method comprises extracting a principal axis of thepoint cloud data to form a coordinate system. The point cloud data istransformed with the coordinate system formed by the extracted principalaxis. The transformed point cloud data is then organized into athree-dimensional voxel structure consisting an X-axis, Y-axis, andZ-axis.

Since the main component of a building are planes, in one or moreembodiments of the invention, a RANSAC plane detection method [Schnabelet al. 2007] is used to detect all the planes existing in the pointcloud data. To avoid the disturbance of small planar objects, onlyplanes with areas larger than an area threshold are selected for furtherinvestigation.

It is assumed that the input point cloud data is registered frommultiple scans of the building interior. In various embodiments, thelaser scanners are mounted on the ground or a platform near the ground,which means that the density of the points on the building floor islarger than the point density on other building elements. Therefore, asimple way to determine the horizontal plane is to find the plane withmost number of points and use the direction of the plane as the Z-axis(i.e. the Z-axis is parallel to the plane containing the greatest totalnumber of laser scanning points).

However, to make the algorithm more robust, a Gaussian sphere map f[α,β]can be generated by counting the number of points on the planes withdifferent normal angles [α,β]. The angle extreme or plane with themaximum number of points is intuitively selected as the Z-axis, with theother two angle extremes or planes comprising different normals beingselected as the directions of the X- and Y-axes. The X- and Y-axes areconstrained to be orthogonal with the Z-axis. In one embodiment, themethod comprises determining one or more planes of the point cloud data.A Gaussian sphere map f[α,β] is generated by counting a total number oflaser scanning points for the one or more planes with a different normalangle [α,β]. The Z-axis is selected to be parallel to a direction of oneor more of the one or more planes with the greatest total number oflaser scanning points. The X-axis and Y-axis are selected to each beparallel to a separate plane with a normal angle orthogonal with thenormal angle of the one or more planes parallel to the Z-axis. Incertain situations, it may be difficult to distinguish the X-axis fromthe Y-axis. However, it is not difficult for a user to manually rotatethe point cloud around the Z-axis for 90 degrees if the X-axis andY-axis need to be swapped. After determination of the three principalaxes, the whole point cloud data can be transformed to make it alignwell with the extracted new coordinate system. An illustrative resultcomparison of a building point cloud before and after principal axisadjustment is shown in FIGS. 4A-4C. FIG. 4A illustrates a front side ofa building before principal axis adjustment. FIGS. 4B and 4C illustratea left side and front side respectively, of a building after principalaxis adjustment.

3D Voxel Construction

Step 302 provides for obtaining point cloud data (to be used forextracting level and reference grid information). In one or moreembodiments of the invention, the building point cloud data used fordatum extraction is acquired by terrestrial laser scanners mounted onthe floors of each level and then combined together. The density nearthe scanner position is usually very dense and then decreases graduallyas the distance increases, which presents a non-uniform density. Incertain embodiments, a histogram is used to detect both level and gridinformation from building point clouds. Direct accumulation of the rawnumber of points biases the histogram toward regions near the scanner.When detecting datum information, area coverage is generally moreimportant than the point sampling number. Therefore, a voxel-basedstructure is more appropriate to re-organize the point cloud data. Sincethe principal direction for the building point cloud has been adjustedas described above, an axis-aligned 3D voxel structure is reasonable andhelpful in representing all of the building points.

At step 304 of FIG. 3, the point cloud data is organized into a 3D voxelstructure. The whole point cloud data is re-organized into small voxelsalong the adjusted X, Y, and Z directions. First, the bounding box ofthe whole point cloud data is subdivided along the X, Y, and Zdirections. The points are then distributed to each voxel. The dimensionof a voxel can be adaptively set to be a multiple of the predefinedthreshold of scanner resolution ε according to the memory capacity. Avoxel with points inside is marked as a “NON-EMPTY” voxel; otherwise, itis an “EMPTY” voxel. This kind of voxel structure is a simplerepresentation of space occupation. As further shown below, it ishelpful to provide a rough estimation of the datum locations.

Multiple Levels Detection

For the point cloud data of building interiors, the location of thelevels are mainly determined by the location of the floors and ceilings.As most floors and ceilings are primarily horizontal, the floor andceiling planes should be perpendicular to the Z-axis after the principalaxis adjustment. Although there are objects and clutter inside thebuilding environment, the region covered by the floor and ceiling shouldstill be relatively large when compared with the region covered by othercluttering objects. To roughly detect the density variations in a pointoccupied area, a height histogram is generated by accumulating theNON-EMPTY voxels along the Z-axis. The peaks in the histogram reveal thelocation of levels (e.g. floors and ceilings).

Rough Level Location Detection

At step 306 of FIG. 3, level information is extracted from the 3D voxelstructure. In one embodiment, the method comprises extracting the levelinformation from a Z-axis histogram of the voxels positioned along theZ-axis of the three-dimensional voxel structure. The voxels along theZ-axis contain at least one laser scanning point and a peak in thehistogram identifies a rough location of the level. To roughly detectthe location of the floors and ceilings, peaks are identified from thehistogram projected along the Z-axis. Although it may be easy tovisually identify the peaks in a small uni-variate histogram, variousembodiments of the invention provided herein allow for automaticallydetecting the peaks from the histogram. As pointed in [Palshikar et al.2009], a data point is a local peak if:

(a) It is a large and locally maximum value within a window, which isnot necessarily a large or global maximum in the entire series; and

(b) It is isolated, i.e., not too many points in the window have similarvalues.

Although many methods have been proposed in [Palshikar et al. 2009] toformalize the notion of a peak, certain embodiments of the inventionutilize the method of an “outlier” detection idea after experimentalevaluation, i.e., a peak should be an “outlier” when considered in thelocal context of a window of multiple points around it (for example 2000points). Let m, s denote the mean and standard deviation of theneighboring data points around x_(i). According to Chebyshev Inequality,a point in the histogram is identified as a peak if:

(i) x_(i)≧m;

(ii) |x_(i)−m|≧hs, for some suitably chosen h>0. The value of h can beset by the user. This criterion guarantees the peak is distinctive frommost points in the neighborhood.

Since only the detection of the locations of floors and ceilings iswanted while large clutter, such as a meeting room table, is to beavoided, in further embodiments of the invention, two more criteria areadded for the peak detection:

(iii) s≧τ_(s), where τ_(s) is a standard deviation threshold, since itis expected that the region area of the level of floor or ceiling wouldvary a lot from its neighboring level;

(iv) x_(i)≧a, where a is an area threshold. Usually it is difficult toset a global area threshold for this condition. In certain embodiments,the area threshold is determined adaptively based on the area A thebuilding point cloud covers. To prevent the disturbance of outliers, thearea the building spans is estimated by projecting the voxel structureto the X-Y-plane and counting the number of NON-EMPTY cells in the 2Dprojection. A cell in the 2D projection is a NON-EMPTY cell if any ofits corresponding 3D voxels is NON-EMPTY. Therefore, it is set α=ε·A. Bydefault, ε is set to 0.5, but this value may be adjusted by the user.

With the above four criteria, peaks will be detected from the histogram.While not all the detected peaks are true peaks, post-processing may beapplied to filter out peaks that are too close together and retain theones with a bigger significance |x_(i)−m|. FIG. 5A illustrates ahistogram of voxels projected along the Z-axis in accordance with one ormore embodiments of the invention. As shown in FIG. 5A, three peaks(bolded lines) are detected for the building point cloud data. As shownin FIG. 5B, these peaks roughly represent the locations of the floorsand ceilings.

Level Location Refinement by Plane Sweeping

At step 308 of FIG. 3, the extracted level information is refined. Sincethe histogram used in the previous section is accumulated with agranularity of the dimension of a voxel, it may still be too coarse foruse. In one or more embodiments, the extracted level location is furtherrefined by plane sweeping.

Each detected rough peak voxel level represents one floor or one ceilinglocated inside the voxel. In this step, the original point count is usedinstead of the represented voxels to check the density variation insidea voxel dimension range. A 2D line-sweeping algorithm is applied todetect the density variation and to find the accurate location of thefloor or ceiling.

Two parallel sweep planes with a small interval are instantiated for theX-Y plane and swept along the Z-axis from the bottom to the top of thepeak voxel level. The number of points located between the parallelsweep planes is counted. Point number changes along the sweeping pathbetween both sides of the sweep plane are accumulated. The extreme valueof the accumulated point number is selected as the position of therefined level location. It should be noted that the interval determinesthe resolution of the level location. As an example, the result of levellocation refinement is shown in FIG. 5B. In one embodiment, the methodcomprises moving two parallel sweep planes defined by the X-axis andY-axis in successive positions along the Z-axis of the point cloud datain a peak voxel. The two parallel sweep planes are separated by aninterval value in the Z-axis direction. The total number of laserscanning points within the interval value of the two parallel sweepplanes is calculated for each position along the Z-axis. The positionalong the Z-axis with the greatest total number of laser scanning pointsis selected as a refined level location.

Orthogonal Reference Grid Detection

After the locations of the floors and ceilings have been detected, thisinformation is then used to segment the whole facility into severallevel regions and to handle the points level by level. This will removethe interference of points from different levels and generate areference grid for each level. Reference grids provide a good referencein building information modeling (BIM) (e.g. using building designsoftware such as Autodesk™ Revit™) for straight wall reconstruction andfloor plan authoring using point cloud data as a reference. Thus, in oneor more embodiments, the building is segmented by level and a referencegrid is generated for each level.

Floor Plan Points

When the segmented point cloud data of each level are projected directlyto the X-Y plane, the floor and ceiling points cover the whole area andhide the location of the wall points. Therefore, in certain embodiments,the horizontal objects which cover big regions such as floor, ceiling,and big tables are removed. By setting a smaller area threshold (asdescribed in the Rough Level Location Detection section), the leveldetection method can also be employed to remove other horizontal regionssuch as tables from the points in one level region. After this step, theinterior structure will appear clearly as shown in FIG. 6. Thus, FIG. 6illustrates the remaining points after floor, ceiling, and largehorizontal objects are removed, in accordance with one or moreembodiments of the invention. However, there are still points ofclutters existing in the remaining projected points. Since the main goalis to detect reference grid lines, the method focuses on wall points.Therefore, the points that do not belong to wall surfaces are furtherfiltered out.

Based on the observation mentioned in [Okorn et al. 2010], the number ofpoints from each height level in the height histogram may varysignificantly as a function of height while the number of wall pointsshould be fairly constant throughout the range. Therefore, it isreasonable to select only some representative cross-section of pointsfor the floor plan points. So, in a second step, a height histogram isrecalculated for the points remaining after the removal of bighorizontal objects. FIG. 7 illustrates a histogram of the number ofvoxels at the floor plan points location in accordance with one or moreembodiments of the invention. The cross-sections are sorted by histogramvalues in increasing order. Cross-sections with bigger height histogramvalues are discarded by floor plan points filtering.

To generate reference grid lines for walls, the focus is mainly onstraight walls. In a third step, the points in the remaining voxels arefurther examined. In one embodiment, for each voxel, the neighboring3*3*3 voxels are obtained, and the points in the neighboring voxels arechecked to see if they form a vertical plane (i.e. parallel to theZ-axis). If yes, this voxel is kept, otherwise it is discarded.

In a fourth step, the remaining voxels are projected onto an X-Y planeand the number of NON-EMPTY voxels is accumulated to form a 2Dhistogram. It is assumed that the detected walls are solid walls, whichmeans more points should be generated during the laser scanning process.Therefore, a threshold is set to filter out the voxels with too fewvertical points.

After these steps, the wall points projected onto the X-Y plane can begenerated, as shown in FIG. 8. These projected wall points show a roughfloor plan structure. In other words, FIG. 8 illustrates floor planpoints after filtering in accordance with one or more embodiments of theinvention.

In one embodiment, the method comprises removing floor and ceilingpoints from the point cloud data. A height histogram of the point clouddata is generated and laser scanning points above a maximum value on theheight histogram are removed. Voxels with neighboring voxels thatconsist of laser scanning points that form a plane parallel to theZ-axis are retained. A two-dimensional X-Y histogram of the retainedvoxels that contain at least one laser scanning point is generated andvoxels with a total number of laser scanning points below a minimumvalue are removed. Wall locations are then extracted from thetwo-dimensional X-Y histogram.

Reference Grid Extraction

At step 310 of FIG. 3, reference grid information is extracted from the3D voxel structure. To extract the reference grids for the straightwalls, a similar histogram technique (as described previously in RoughLevel Location Detection section) is used to extract the reference gridstructure. The only differences are that the floor plan points areprojected on 2D space and the reference grids are extracted along boththe X-axis and Y-axis. The following steps are employed to extract thereference grid lines:

1) The floor plan points are organized into a 2D grid structure in thefirst step and points are discretized into the cells. The reason issimilar to what was explained in the 3D Voxel Construction section: Dueto the laser scanner sampling density difference, counting the absolutenumber of floor plan points will directly bias the histogram;

2) The number of NON-EMPTY cells in each row of the 2D grid is countedand a histogram is formed along the Y-axis. The local peaks in thehistogram are detected using the method described above (Rough LevelLocation Detection). In one embodiment, the Y-axis histogram comprisesvoxels along the Y-axis that contain at least one laser scanning point.A peak in the Y-axis histogram identifies a rough wall location orlocation marker of the reference grid. After that, the peak value foreach local peak row is refined using the method described in LevelLocation Refinement by Plane Sweeping section (step 312 of FIG. 3). Theonly difference is that a line sweeping method is used instead of aplane sweeping method here to refine the extracted reference gridinformation. The detected peaks are regarded as the locations of thereference grids along the Y-axis. In one embodiment, two parallel sweeplines are moved in successive positions along the Y-axis of the pointcloud data in a peak voxel. The two parallel sweep lines are separatedby an interval value in the Y-axis direction. The total number of laserscanning points within the interval value of the two parallel sweeplines is calculated for each position along the Y-axis. The positionalong the Y-axis with the greatest total number of laser scanning pointsis also selected as a refined wall location on the reference grid;

3) The number of NON-EMPTY cells in each column of the 2D grid iscounted and a histogram is formed along the X-axis. Then the same methoddescribed above (step 2) is used to extract and refine the locations ofthe reference grids along the X-axis. In one embodiment, the X-axishistogram comprises voxels along the X-axis that contain at least onelaser scanning point. A peak in the X-axis histogram identifies a roughwall location or location marker of the reference grid. In a furtherembodiment, two parallel sweep lines are moved in successive positionsalong the X-axis of the point cloud data in a peak voxel. The twoparallel sweep lines are separated by an interval value in the X-axisdirection. The total number of laser scanning points within the intervalvalue of the two parallel sweep lines is calculated for each positionalong the X-axis. The position along the X-axis with the greatest totalnumber of laser scanning points is also selected as a refined walllocation on the reference grid;

A final result of reference grids of floor plans is shown in FIG. 9. Inthis example, the reference grids were created in Autodesk™ Revit™.

Embodiments of the invention provide a method to extract the levelinformation and orthogonal reference grids for floor plans of a buildingscanned by terrestrial laser scanners. The validity of this method hasbeen evaluated (e.g., on the Autodesk™ Revit™ platform) as shown in FIG.10. In this regard, FIG. 10 illustrates extracted datum information fromthe building's point cloud in accordance with one or more embodiments ofthe invention. The extracted datum information provides an overview ofthe level and floor plan information of the building interior and helpswith reconstructing a detailed blueprint of the building from pointcloud data.

Building retrofit is a big market and is still rapidly expanding. Moreand more required output has been shifted from 2D drawings to 3D BIMs,which means that there is a strong need for BIM software in creatingas-built models from laser scanning data for existing buildings.Specifically, the creation of blueprints of the existing conditions ofbuildings is a common task in the AEC domain. Currently, it isunderstood that no intelligent floor plan generation tools have beenprovided in existing commercial software. The method provided hereinallows for automatic extraction of datum information including levelsand orthogonal reference grids from laser scanning data of buildinginteriors, which provides helpful information that aids blueprintcreation and reduces a user's labor in the reconstruction process.

CONCLUSION

This concludes the description of the preferred embodiment of theinvention. The following describes some alternative embodiments foraccomplishing the present invention. For example, any type of computer,such as a mainframe, minicomputer, or personal computer, or computerconfiguration, such as a timesharing mainframe, local area network, orstandalone personal computer, could be used with the present invention.

The foregoing description of the preferred embodiment of the inventionhas been presented for the purposes of illustration and description. Itis not intended to be exhaustive or to limit the invention to theprecise form disclosed. Many modifications and variations are possiblein light of the above teaching. It is intended that the scope of theinvention be limited not by this detailed description, but rather by theclaims appended hereto.

REFERENCES

-   [Adan et al. 2011] Antonio Adan, Xuehan Xiong, Burcu Akinci, and    Daniel Huber. Automatic Creation of Sementiacally Rich 3D Building    Models from Laser Scanner Data, Proceedings of the International    Symposium on Automation and Robotics in Construction (ISARC), June,    2011;-   [Huber et al. 2010] Daniel Huber, Pingbo Tang, Antonio Adan, Brian    Okorn, and Xuehan Xiong. Using Laser Scanners for Modeling and    Analysis in Architecture, Engineering, and Construction. In    Proceedings of the Conference on Information Sciences and Systems    (CISS), Princeton, N.J., 2010;-   [Dorninger et al. 2008] Dorninger, P. and Pfeifer, N. A    comprehensive automated 3D approach for building extraction,    reconstruction and regularization from Airborne Laser Scanning Point    Clouds, Sensors 2008, Vol. 8, 7323-7343;-   [Palshikar et al. 2009] Girish Keshav Palshikar, 2009, Simple    Algorithms for Peak Detection in Time-Series, In Proceedings of 1st    IIMA International Conference on Advanced Data Analysis, Business    Analytics and Intelligence, Ahmedabad, India, Jun. 2009;-   [Li et al. 2011] Li, W., Wolberg, G., and Zokai, S., 2011,    Lightweight 3D Modeling of Urban Buildings from Range Data. In:    3DIMPVT IEEE (2011), p. 124-131;-   [Nguyen et al. 2005] Nguyen, V. Gachter, S. Martinelli, A.    Tomatis, N. Siegwart, R. A Comparison Of Line Extraction Algorithms    Using 2d Range Data For Indoor Mobile Robotics, In Proc. of the    IEEE/RSJ International Conference on Intelligent Robots and Systems,    IROS, 2005, p. 1929-1934;-   [Okorn et al. 2010] Okorn B., Xiong X., Akinci K. and Huber D, 2010,    Toward Automated Modeling of Floor Plans, in. Proceedings of the    Symposium on 3D Data. Processing, Visualization and Transmission,    Paris, France, 2010;-   [Schnabel et al. 2007] Schnabel, R., Wahl, R. and Klein, R. 2007,    Efficient RANSAC for Point-Cloud Shape Detection. Computer Graphics    Forum, 26: 214-226;-   [Schindler et al. 2003] Schindler, K., and Bauer, J. 2003. A    model-based method for building reconstruction. In Proc. of IEEE    Workshop on Higher Level Knowledge in 3D Modeling and Motion    Analysis, 74; and-   [Schroter et al. 2002] Schroter, D. Beetz, M., and Gutmann, J.-S.,    2002, RG mapping: Learning compact and structured 2d line maps of    indoor environments, in: Proceedings of the IEEE International    Workshop on Robot and Human Interactive Communication, ROMAN'02,    2002.

What is claimed is:
 1. A computer-implemented method for extractinglevel and reference grid information from point cloud data, comprising:obtaining point cloud data comprising laser scanning points; organizingthe point cloud data into a three-dimensional structure of voxels, thethree-dimensional structure consisting of an X-axis, Y-axis, and Z-axis;extracting level information; and extracting reference grid informationfrom an X-axis histogram of the voxels positioned along the X-axis ofthe three-dimensional voxel structure and a Y-axis histogram of thevoxels positioned along the Y-axis of the three-dimensional voxelstructure.
 2. The computer-implemented method of claim 1 wherein: thepoint cloud data is obtained using a laser scanner; and the point clouddata comprises laser scanning points for a building.
 3. Thecomputer-implemented method of claim 1 further comprising: extracting aprincipal direction of the point cloud data by determining X-axis,Y-axis, and Z-axis directions of the point cloud data; and transformingthe point cloud data such that a front face of the building is parallelto a plane defined by the X-axis and Z-axis or a plane defined by theY-axis and Z-axis; wherein the transformed point cloud data is organizedinto the three-dimensional structure of voxels.
 4. Thecomputer-implemented method of claim 1 further comprising determiningone or more planes of the point cloud data wherein the Z-axis of thepoint cloud is parallel to the plane containing the greatest totalnumber of laser scanning points.
 5. The computer-implemented method ofclaim 1 further comprising: determining one or more planes of the pointcloud data; generating a Gaussian sphere map f[α,β] by counting a totalnumber of laser scanning points for the one or more planes with adifferent normal angle [α,β]; selecting the Z-axis to be parallel to adirection of one or more of the one or more planes with the greatesttotal number of laser scanning points, and the X-axis and Y-axis to eachbe parallel to a separate plane with a normal angle orthogonal with thenormal angle of the one or more planes parallel to the Z-axis.
 6. Thecomputer-implemented method of claim 1 further comprising: extractingthe level information from a Z-axis histogram of the voxels positionedalong the Z-axis of the three-dimensional voxel structure; wherein thevoxels along the Z-axis contain at least one laser scanning point; and apeak in the histogram identifies a rough location of the level.
 7. Thecomputer-implemented method of claim 6 wherein a point x_(i) in thehistogram is identified as a peak if: i) x_(i) is greater than or equalto the mean m of neighboring points around x_(i); ii) an absolute valueof x_(i) minus m is greater than or equal to the standard deviation s ofthe neighboring points around the peak x_(i) multiplied by a value hpredefined by the user; iii) s is greater than or equal to a standarddeviation threshold τ_(s); and iv) x_(i) is greater than or equal to anarea threshold a.
 8. The computer-implemented method of claim 1 furthercomprising refining the extracted level information by plane sweeping.9. The computer-implemented method of claim 8 further comprising: movingtwo parallel sweep planes defined by the X-axis and Y-axis in successivepositions along the Z-axis of the point cloud data in a peak voxel, thetwo parallel sweep planes separated by an interval value in the Z-axisdirection; calculating the total number of laser scanning points withinthe interval value of the two parallel sweep planes for each positionalong the Z-axis; wherein the position along the Z-axis with thegreatest total number of laser scanning points is selected as a refinedlevel location.
 10. The computer-implemented method of claim 1 whereinthe building is segmented by level and a reference grid is generated foreach level.
 11. The computer-implemented method of claim 10 wherein: theX-axis histogram comprises voxels along the X-axis that contain at leastone laser scanning point and the Y-axis histogram comprises voxels alongthe Y-axis that contain at least one laser scanning point; and a peak inthe X-axis histogram or Y-axis histogram identifies a rough walllocation on the reference grid.
 12. The computer-implemented method ofclaim 1 further comprising refining the extracted reference gridinformation by line sweeping.
 13. The computer-implemented method ofclaim 12 further comprising: moving two parallel sweep lines insuccessive positions along the X-axis of the point cloud data in a peakvoxel, the two parallel sweep lines separated by an interval value inthe X-axis direction; calculating the total number of laser scanningpoints within the interval value of the two parallel sweep lines foreach position along the X-axis, wherein the position along the X-axiswith the greatest total number of laser scanning points is selected as arefined wall location on the reference grid; moving two parallel sweeplines in successive positions along the Y-axis of the point cloud datain the peak voxel, the two parallel sweep lines separated by an intervalvalue in the Y-axis direction; and calculating the total number of laserscanning points within the interval value of the two parallel sweeplines for each position along the Y-axis, wherein the position along theY-axis with the greatest total number of laser scanning points is alsoselected as a refined wall location on the reference grid.
 14. Thecomputer-implemented method of claim 1 further comprising: removingfloor and ceiling points from the point cloud data; generating a heighthistogram of the point cloud data and removing laser scanning pointsabove a maximum value on the height histogram; retaining voxels withneighboring voxels that consist of laser scanning points that form aplane parallel to the Z-axis; generating a two-dimensional X-Y histogramof the retained voxels that contain at least one laser scanning pointand removing voxels with a total number of laser scanning points below aminimum value; and extracting wall locations from the two-dimensionalX-Y histogram.
 15. An apparatus for extracting level and reference gridinformation from point cloud data in a computer system comprising: (a) acomputer having a memory; and (b) an application executing on thecomputer, wherein the application is configured to: (1) obtain pointcloud data comprising laser scanning points; (2) organize the pointcloud data into a three-dimensional voxel structure consisting of anX-axis, Y-axis, and Z-axis; (3) extract level information; and (4)extract reference grid information from an X-axis histogram of thevoxels positioned along the X-axis of the three-dimensional voxelstructure and a Y-axis histogram of the voxels positioned along theY-axis of the three-dimensional voxel structure.
 16. The apparatus ofclaim 15 wherein the application is further configured to: extract aprincipal axis of the point cloud data to form a coordinate system;transform the point cloud data with the coordinate system formed by theextracted principal axis; and organize the transformed point cloud datainto a three-dimensional voxel structure consisting an X-axis, Y-axis,and Z-axis.
 17. The apparatus of claim 15 wherein the application isfurther configured to determine one or more planes of the point clouddata wherein the Z-axis of the point cloud is parallel to the planecontaining the greatest total number of laser scanning points.
 18. Theapparatus of claim 15 wherein: level information is extracted from aZ-axis histogram of voxels projected along the Z-axis of thethree-dimensional voxel structure; the Z-axis histogram comprises voxelsalong the Z-axis that contain at least one laser scanning point; and apeak in the histogram identifies a rough location of a level.
 19. Theapparatus of claim 15 wherein: the X-axis histogram comprises voxelsalong the X-axis that contain at least one laser scanning point and theY-axis histogram comprises voxels along the Y-axis that contain at leastone laser scanning point; and a peak in the X-axis histogram or Y-axishistogram identifies a rough location marker of the reference grid. 20.The apparatus of claim 15 wherein the building is segmented by level anda reference grid is generated for each level.